Although solid-state image pickup devices such as CCD image pickup device are used in such image pickup apparatus as TV camera, video camera, electronic camera, etc., there is a problem that the dynamic range of a solid-state image pickup device is much narrower than that of a silver salt photographic film.
To eliminate this problem, there have been proposed techniques in which an image having an increased dynamic range is obtained by synthesizing two image pickup signals of different exposure amount read out from a single image pickup device. For example, Japanese patent application laid open No.4-207581 discloses those constructed as follows.
In particular, one of those disclosed therein comprises an image pickup device including a light receiving section having a plurality of light receiving elements for photoelectric conversion of optical information and a means for transferring signals from the light receiving section. Immediately after performing a first exposure at the light receiving section and transmitting signals of the first exposure to the transfer means, a second exposure having a longer exposure time than the first exposure is started. After outputting signals of the first exposure from the image pickup device, signals of the second exposure are transmitted to the transfer means to output signals of the second exposure from the image pickup device. The second exposure is thereby performed in the period during which signals of the first exposure are outputted from the image pickup device so as to make smaller a temporal difference in picture taking caused by such plurality of times of exposure. In this aspect, a disclosure is also made with respect to one in which the first exposure time is controlled by transmitting of signals due to the exposure from the light receiving section to the transfer means, i.e., by an electronic shutter function, while the second exposure time is controlled by a means for shutting off light to control an exposure time of the image pickup device. The second exposure time can thus be controlled independently from the transferring operation of the first exposure signals.
Further, one as shown in FIG. 1 is disclosed in Japanese patent application laid open No.6-141229 as a construction for synthesizing two image pickup signals that are different in exposure amount to synthesize an image having a wide dynamic range. In particular, signals with a shorter charge accumulating period CS1 read out from CCD 501 are recorded at memory 502, and the recorded signals are read out from the memory 502 at substantially the same timing as readout from CCD 501 of signals with a longer charge accumulating period CL1. The signals read out from the memory 502 are multiplied by CL1/CS1 for example “4” using a multiplier 503. By this multiplication using such factor, signal levels of those signals differing from each other in charge accumulating period with respect to the same object are theoretically brought to the same level.
Next, a level weighting means H 505 assigns a weight corresponding to signal level to the signal of charge accumulating period CS1 from the multiplier 503. A weight is assigned also to the signal of charge accumulating period CL1 at a level weighting means L 504. The characteristics of the respective weights are shown in FIG. 2 where abscissa represents an input signal level and ordinate represents a weighting factor to be assigned to the signals. The level weight L is a weight of 1 for input levels up to 80%, is linearly lowered in weight value for 80% to 100% and puts the weighting factor to 0 at the input level of 100%. The level weight H on the other hand, is a weight of 0 for levels up to 80%, is linearly increased in weight value for 80% to 100% and puts the weighting factor to 1 at the input level of 100%. The weighting factor is always 1 for input levels of 100% and above. The image signals differing from each other in dynamic range are weighted thus and are synthesized as added together at an adder 506 to form a single piece of image. The portions in good conditions (with favorable S/N and without saturation) are extracted through this weighting process to synthesize an image having a wide dynamic range. Since the synthesized image is processed at a rate about twice the normal when signals are read out from CCD 501, the scanning rate of signals is converted at a rate converter 507 to 1/2 so that it corresponds to a standard TV signal. Further, while the signal level of the synthesized image at its maximum is 400%, the level is compressed to 100% at a level compression means 508. It should be noted that numeral 509 in FIG. 1 denotes a timing control section.
In this manner, a difference in signal level due to difference in charge accumulating period is corrected with respect to the two signals that are different in charge accumulating period. A larger weight is assigned to those portions where signals are not saturated while an adequate S/N is kept to make a selection from these image signals and at the same time to smoothly synthesize the two image signals. The signal obtained as having a wide dynamic range and an increased signal amplitude is then outputted after compressed to a standard signal level. An image having an increased dynamic range is thereby synthesized from the two image signals having been controlled with respect to their charge accumulating periods at CCD of which dynamic range is narrow.
Further, Japanese patent application laid open No.62-108678 discloses a technique in which, when an image having a wide dynamic range is to be obtained by synthesizing two image pickup signals that are different in exposure amount, one of the image pickup signals is picked up by using only natural light while the other image pickup signal is generated by a picture taking process using a strobe light. These image pickup signals are compared with each other and the portions of greater signal are outputted to obtain a synthesized image.
Furthermore, a disclosure is made in Japanese Patent Publication No.2659570 with respect to an electronic still camera in which, at the same time of causing strobe light to emit for a plurality of times, a plurality of times of readout of charge are effected within a time period for taking one frame of picture so that amount of light to be received is adjusted by control of an emission amount of strobe light or an electronic shutter operation of solid-state image pickup device so as to increase dynamic range without a saturation in output of the solid-state image pickup device even in high luminance regions.
Referring now to the above Japanese patent application laid open No.4-207581, it discloses a technique where, of two image pickup signals that are different in exposure amount, an image of smaller exposure amount is picked up first in time by using an electronic shutter function of the image pickup device and, then, an image of greater exposure amount is picked up while controlling the charge accumulating time using a means for shutting off light (mechanical shutter). If, however, a means for shutting off light is employed in an electronic camera, such means for shutting off light is normally kept in its opened state. A charge accumulating time is controlled by a transition from the opened state to its closed state. In this case, therefore, the charge accumulating time differs from one position to another on the light receiving surface of the image pickup device. A shading thus results. In the disclosure of the above publication, however, this point is not taken into consideration at all and a means for preventing a shading is not mentioned either. There is another problem that, in respect of control of charge accumulating period, a means for shutting off light is not very accurate when compared to an electronic shutter of the image pickup device.
Furthermore, in the techniques disclosed in the above publications, an accuracy is not taken into consideration in setting an exposure amount ratio of two image pickup signals when the two image pickup signals of different exposure amount are to be obtained. The problems in the case of using a flash emission means are not mentioned either.
In particular, if the luminance of the object of picture is at a relatively low level, picture taking is performed by using a strobe light in combination. In forming a synthesized image having a wide dynamic range through a synthesis of two image pickup signals of different exposure amount, if an attempt is made to simply synthesize the obtained two image pickup signals where one of the image pickup signals is generated by using natural light alone and the other of the image pickup signals by a strobe light in combination, it is difficult due to the intrinsically large variance of a strobe light, even with a constant exposure time, to attain a precise correspondence between the two image pickup signals in order to achieve a predetermined ratio of their exposure amounts. Further, in the case of using a strobe light, while image pickup signal having a predetermined exposure ratio in relation to the image pickup signal picked up by natural light alone can be obtained at the portions within a picture frame reached by the strobe light, image pickup signal in the portions not reached by the strobe light is of the same exposure amount as one picked up by natural light alone. The predetermined exposure ratio cannot be obtained. Furthermore, in the case of synthesizing image pickup signal by natural light and image pickup signal by a strobe light, since these are the image pickup signals different from each other in white balance, there is a problem that color balance is partially lost in the synthesis. Moreover, in the case of image pickup apparatus such as an electronic camera including an exposure setting means (AE), a white balance setting means (WB), a gain control means, a stop control means, etc., when two image pickup signals of different exposure amount are synthesized to obtain an image having a wide dynamic range, no consideration is taken as to how to achieve a correspondence between them.
Further, while the above Japanese patent application laid open No.6-141229 discloses a technique in which synthesis is smoothly performed while changing the weighting factor when two image pickup signals of different exposure amount are synthesized to synthesize an image having a wide dynamic range, the problem of false color in synthesizing a color image is not considered and no disclosure is made with respect to the problems in using a strobe.